On the computation of Shannon Entropy from Counting Bloom Filters

A Bloom filter [Blo70] is used to perform set membership testing without having the actual data in a set available. The test can give false positives (outputs that the tested element is in the set while it is not), but never false negatives (if the test tells the element is not in the set, it definitely is not). The data structure consists of an array, which is initially set to all zeros, and a set of hash functions. The outcome range of the hash functions is exactly as large as the length of the array. To add an element to the structure (i.e., register it as en element of the set), the element is hashed with each of the hash functions and the bits at the indexes which corresponds to the outcome of the hash function applications is set to 1. To test whether an element is in the set, the same hash functions are applied on the element and each of the bits at indexes corresponding to their outcomes is checked. If any of these bits is set to 0, then is concluded that the element was not in the set. This is correct because if it had been there, all bits would have been set to 1 at an earlier point. If all these bits are set to 1, it is concluded that the element might be in the set. Either, the element is really in the set and hence its bits were set to one at an earlier point, or all of these bits were set to 1 by the application of the hash functions on other elements which are in the set. In the latter case we consider the outcome a false positive. A drawback of Bloom filters is that it is impossible to remove any elements from it. One might naively think that one could remove an element by setting all bits corresponding to the outcome of the hash functions to 0. However, this might violate the properties described above in case any of these bits was also set to 1 by any hash function application on any other element which is still in the set. To overcome this issue Li Fan et al. [FCAB00] introduced counting Bloom filters. A counting Bloom filter works like a normal Bloom filter, but instead of maintaining a bit array, an array with integers, initially filled with zeros, is used. When an element is added to the filter, the hash functions are computed and instead of setting the corresponding bits to 1, the count at these indexes is incremented with 1. The test for membership will conclude that the element is not in the set in case the counts at any of the hash function outcome indexes is 0.